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Page 5, after line 19, insert the following list that currently appears on pages 43-44 
of the original specification: 

Advantageous features of preferred exemplary embodiments include: 

• Identifying coordinates on 2D image to be mapped to triangles that form a 3D 
face; 

• Still image editor for user of a video game; 

• Image editor that allows editing of image in 2D and 3D modes; 

• 2D image editing while 3D mapped image is being displayed in real time to 
show effect of editing; 

• Selection of various 3D heads on which to map a 2D image, and manipulation 
of those 3D heads to improve appearance of 2D face mapped on the head; 

• 3D head manipulation of both front view shape and front-to-back distance of 
head; 

• Portable storage of a personalized game player; 

• Random placement of personalized game player faces onto computer 
controlled players, such as guards; 

• Mapping of 2D image on 3D face; 

• Using a digital camera mounted on a hand controller to take pictures for a 
game console; 
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• Using a digital camera mounted on one hand controller where a 2D image - 
capture memory resides in the digital camera, and a memory for storing a 3D 
head mapped with the 2D image is attached to the same or another hand 
controller; 



• Personalize game player having a face of the user. 



The paragraph beginning at page 7, line 18: 




FIGURE 15 is an exemplary video game screen image showing a 2D facial image 


mapped on a head and a prompt to save this personalized head; 


The paragraph beginning at page 8, Une 9: 




FIGURES 20 and 21 are exemplary video game screen images of an image editor 


having controls for manipulating the shape of a 3-D head, and 


The paragraph beginning at page 8, line 14: 



Fig. 1 is a block diagram of a game system according to one embodiment of the 
present invention. The machine is, for example, a game machine 10 (e.g., the product 
"Nintendo 64") utihzing an advanced technology, such as 32 or 64 bits, which is high in 
processability (i.e., processing capability , for example, in CPU bit number, CPU 
program processing capability per unit time or image representability). The machine uses 
a removable memory cartridge 12 (hereinafter referred to as "cartridge") as one example 
of an external memory medium for storing game programs, and other data. 
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The paragraph beginning on page 9, line 1 : 



Controllers 16a, 16b, such as hand controllers, are electrically connected to the 
machine 10 typically via wires 17, although a wireless connection may be used. The 
hand controller has a joystick, buttons, keys and other user input devices for interactively 
controUing game play as displayed on a television 18. In addition, the hand controller 
includes an electrical connector 24 which provides a data connection between a 
removable data transfer pack 26 plugged into the controller and a processor in the game 
machine. A transfer cartridge is disclosed in pending U.S. Patent Application Serial No. 
09/189,797, entitled "Game System Operable With Backup Data On Different Kinds of 
Game Machines" and filed November 12, 1998, and the entire contents of which is 
incorporated by reference. The data transfer pack includes a connector 28 for electrically 
connecting to other game cartridges, a memory pack cartridge or to a cartridge with 
digital camera 14, as is disclosed in pending U.S. Patent Appln. Serial No. 09/430,169, 
entitled "Portable Game Machine Having Image Capture, Manipulation and 
Incorporation" filed October 29, 1999, the entire contents of which is incorporated by 
reference. The digital camera cartridge 14 may be used in connection with the transfer 
cartridge 26 to interface the digital camera cartridge to the machine, via the transfer pack 
and controller. An advantage of inserting the digital camera cartridge into a hand-held 
controller is that a user may easily align the field of view of the camera with a desired 
subject simply by holding and positioning the controller and camera. In addition, a 
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digital camera cartridge may be inserted into a connector of the controller, while at the 
same time a game cartridge 12 having an image editor and game program may be 
^ inserted in a cartridge connector of the machine. In addition, the digital camera may be 
A inserted into a first hand controller 16a, and a second hand controller 16b may be used to 
input user control information for capturing, editing and saving an image and to connect 
to a memory pack (not shown in Figure 1) for storing data regarding the image and 
associated personalized 3D head of an animated game player. 



The paragraph beginning on page 10, hne 14: 



FIGURE 2 is an exploded view showing the structure of a camera cartridge 14. 
The camera portion is integrally formed on the upper portion (upper edge) when viewed 
from the front of the housing 50a, 50b, which contains an electrical circuit having 
electronics to support the digital camera, memory for storage of 2D digital images and a 
connector 54 for coupling the camera cartridge to a game machine and data pack. The 
housing is formed, for example, in a flat, vertically elongated rectangular parallelepiped 
shape having a prescribed thickness. The connector 54 is provided inside an opening in 
the cartridge housing 50 a, b, opening for contacting with a terminal (e.g., edge 
connector) of a substrate 52 included in the cartridge to electrically connect a circuit on 
the substrate to an electric circuit in the housing. The camera cartridge 14 includes a 
generally spherical camera housing 20a, 20b containing an image pickup device, e.g., 
lens 22 and CCD 30, and is supported rotatably (in a lateral direction when viewed from 



-5- 



594175 



STAMPER et al. 
Serial No. 09/854,942 



the front by the supporting portion) on a mounting 32a, 32b on the housing of the 
cartridge. 



The paragraph beginning on page 11, line 21: 



In this manner, the supporting portion rotatably supports the camera portion in a 
lateral direction of the body portion to allow the user to easily change the shooting range 
by simply changing the direction of the camera portion and also to shoot and display the 
forward direction of the user and the user himself/herself (or the backward direction of 
the user) on the image display device when the rotation range is selected to 
approximately 180 degrees. Further, the supporting axis with which the supporting 
portion supports the camera portion is inclined to allow the user to freely adjust the 
shooting range in the forward direction or on the user side by only slightly including both 
wrists when holding the housing with both hands, without requiring the user to take an 
uncomfortable position when adjusting, thereby reducing the fatigue of the wrists. 



The paragraph beginning on page 12, line 14: 



The game machine 120 includes a CPU 121 that is connected to an input/output 
interface 122 (hereinafter referred to as "I/O"). To the I/O are connected connectors 
123b, 123d for connection with a hand controller 124. Where the game machine 120 is 
usable for a game in which a plurality of players participate at a same time, a plurality of 
sets of connectors and controllers are provided. The hand controller 124 may also 
include a connection 150 to a portable memory pack 115, having a read/write memory for 
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storage of user data, e.g., facial images, to be used during game play. The memory 
^ cartridge connectable to the hand controller incorporates, by mounting on a substrate, a 
non-volatile memory (e.g., ROM, EP-ROM; hereinafter referred to as "ROM") for 
storing image files, editing files and identification codes. 

The paragraph beginning at page 13, line 15: 

FIGURE 4 is a block diagram of the components for capturing an image of a 
subject 300, e.g., a face of a person. Light reflected off of the subject towards the digital 
camera cartridge 304 is focused by a lens 302 on a charged-coupled-semiconductor 
device (CCD) 306. The CCD generates electrical signals representing an image of the 
subject. An analog-to-digital converter 308 converts the electrical signals into a digital 
image of the subject. The digital image is formatted by another converter 310 into a 
digital file that is stored in a RAM memory 312 in the camera cartridge 304. The 
memory 312 may save several digital image files representing different pictures. In 
addition, these image files can be written into memory over existing files, so as to erase 
those overwritten image files. 

The paragraph beginning on page 16, line 3: 

During image capture, an oval template 1004 is displayed on the screen to prompt 
the user to center his face in image capture area. During the "Take Picture" menu 
0[O (Fig. 10) the current visual image as being received by the camera is displayed on screen, 
and is updated by the camera at around 2 times per second. A blue transparent overlay 
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template 1004 highlights a vertical oval area of the image which is deemed to be the ideal 
position for the user to position his or her face. Face images that fit correctly with this 
oval will need little fine tuning and as such the whole capture/editing process is speeded 
up by properly positioning the user's face in the oval. The user moves before the camera 
so that his head is centered in the oval template shown in real-time on the screen. 

The paragraph beginning on page 17, line 7: 



A facial image may also be selected by choosing one of the stored 2D images in 
the camera cartridge (Fig. 9) or by choosing a personalized 3D animated game player 
stored in a game pack or other memory device coupled to the game pack, controller or 
machine. If a personalized 3D head is selected (S64), the user may be first prompted to 
identify the game mode, e.g., multi-player combat, single player missions or both, in step 
S66. The user is then presented on the display with the 3D personalized heads for 
selection in S68. The user may then begin game play or edit (S70, S72, S74) the existing 
head. 



The paragraph beginning at page 17, line 21: 

Once a pre-existing image is selected, the image is transferred from the camera 
memory to memory in the game machine, in step S80. Regardless of whether the image 
is a pre-existing image or an image captured from a real-time camera image, the program 
saves the image in temporary memory in the game machine so that the image may be 
applied to a head and edited, in step S82. The mapping of a 2D image onto the head may 
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occur during start-up of each game (as the existing heads are readout of the memory 
pack) and, during the editing process, as 3D heads and 2D facial images are being edited. 
The major portions of each 3D head are its face, hair and sides (neck and sides around the 
ears). The two-dimensional facial image 1200 is mapped to the face portion 1204 of the 
3D head 1206, as is shown in HGURE 12. The hair 1206 and sides 1208 of the head are 
not mapped with a 2D image. The 3D face area 1202 is a 3D generally oval surface. 

V 

This face area surface is defined mathematically by computer software as triangles, e.g., 
20 triangles, that when assembled together form a 3D surface of the face of the head. To 
generate a 3D face, to be inserted in a head that may be oriented from the front, side, 
looking-up, looking down or other point of view, the computer determines the location 
and position of each of the triangles that define the face. The location in three- 
dimensions of the surface of the face is determined, determining the location of each of 
the mathematically defined triangles that form the 3D face image. 

The paragraph beginning at page 18, Une 18: 



The 2D facial image is applied to the 3D face by mapping the 2D image to the 
triangles that form the 3D face. For mapping, the 2D image is segmented into triangular 
sections that correspond to the 3D triangles of the face. The segmentation is done by 
^ ^'X^ identifying three points on the 2D image that correspond to the comers of each triangle of 
the 3D face. The section of the 2D image within the triangle defined by each of the three 
identified points is mapped onto the corresponding triangle of the 3D face. For example. 
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one of the triangles of the 3D face portion may correspond to the nose of the face. To 
map the nose of the 2D image to the 3D nose triangle, three points on the 2D image are 
identified that outUne the nose shown in that 2D image. The 2D image within the 
^ 1 3 triangle defined by the three selected points are mapped to the triangle corresponding to 
the nose of the 3D face. Similarly, the other triangular sections' 2D image are mapped to 
the other triangles that make up the 3D face. The mapping of the 2D image onto a 3D 
face is accomplished each time the video game starts up and during editing. 

The paragraph beginning at page 20, line 24: 



To calculate the transformed (texture) co-ordinates, equations 4 and 5 are applied to each 
vertex, S1310, S1312, S1314 and S1316. 



The paragraph beginning at page 21, line 8: 



The stored 3D "heads" which the user selects are stored in the game program (or 
memory controller pack) as "slots" S82. When completed, the one slot contains all the 
information to generate a custom character head. This includes face texture information, 
hair and face colors, head type, etc. The game allows multiple slots for multiple heads 
that can be used in the single player version of the game where the custom heads can be 
used for either your player character's head or that of a random guard or other computer- 
controlled game character. 
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The paragraph beginning at page 22, Hne 6: 



Once a facial image has been selected, the user is prompted to apply the image to 
an animated three-dimensional head, in step 568 and shown in FIGURE 15. The machine 
may have stored several, e.g., five, animated 3D heads which are used to create animated 
game players. These heads may be male or female, and have different shapes, hair color 
and skin tone. The heads are presented on the display as individual 3D head images. The 
heads may have blank faces, a computer generated face or have the face of the 2D facial 
image that has been captured and selected for use in creating a personalized head. The 
heads may move, e.g., turn from side to side and tilt up and down, so that the user may 
see the head from different angles. The user selects a head on which to apply the facial 
image that has been captured. 



11 



The paragraph beginning at page 23, line 14: 



To edit an animated 3D head with a mapped 2D facial image, the user selects one 
of several available editing functions, including select a new 2D facial image S86, select 
a new 3D head S88, reposition the 2D facial image S90, adjust the 3D shape of the head 
S92, change the hair color and skin tone of the head S94, delete the saved head S96, 
cancel any changes S98 that have been made to a saved head during the editing process, 
and save any changes made to the head and facial image SI 00. 
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The paragraph beginning at page 23, line 21: 



In particular, the editor contains various sections to alter the characteristics of the 
current head, these including: 

Take Another Picture (S86) 

This option leads to the "Take Picture" menu allowing the user to change the 
image that is mapped onto the head's face. 
Change Head (S88) 

The "Change Head" function allows the user to alter his or her choice of the 
special face mapping enabled character heads. 
Position Picture (S90) 

This menu allows the user to change which parts of the captured image are 
mapped onto the head's face. Five control handles allow the size, shape and 
position of the aforementioned oval area to be altered, as is shown in FIGURE 19. 
The software maps the 2D image that falls inside this oval onto the facial area of 
the head. An additional "Brightness" feature allows the user to change the 
brightness of the facial image helping to disguise the joint between the face and 
the rest of the head. 



The paragraph beginning at page 24, line 15: 

The editing by the user of the captured 2D image S90 includes: adjusting the 
position of the oval template over the 2D image, expanding and contracting the oval 
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shape about the major and minor axes (length and width) of the oval, and adjusting the 
brightness of the 2D image 1900 in Fig. 19. The oval template 1902 defines the portion 
of the 2D image that will be mapped to the 3D face. The template is translucent so that 
. C the entire 2D image (both inside and out of the template) may be viewed - which 

facilitates the positioning of the template on the 2D image. The template can be moved 
with respect to the 2D image by use of cursor commands and drag points that are on the 
rim of the oval and that can be manipulated by the user with the hand controller and a 
cursor that appears on the image in Figure 19. 

The paragraph beginning at page 25, line 3: 

Moving the template with respect to the image allows the user to better ahgn his 
face in the oval template and, thereby, adjusts the alignment of the facial image on the 3D 
head 1904. Expanding and contracting the length and width of the oval also allows a user 
to better position his facial image within the oval and, thus, onto the 3D face. 

The paragraph beginning at page 25, line 19; 

The 2D image editor may also automatically balance the contrast of the image to 
reduce the unintended effects of shading on the 2D image. These shading effects can 
cause poor 3D visual effects when the 2D image is mapped on the 3D face. To balance 
the contrast, the image editor compares the brightness of the left side of the image to the 
right side of the image. If the brightness between the two sides varies excessively, such 
as by more than 10%, then the editor reduces the brightness on the bright side of the 
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image and may increase the brightness on the dark side of the image. In addition, the 
brightness/darkness adjustments are more pronounced at the edges of the oval image than 
j]^ iX the center because the brightness adjustments are apphed in a linear manner in which 
the center of the image is not adjusted so as to avoid creating a perceptible contrast 
change at the center of the image. 

The paragraph beginning at page 26, line 8: 

When using a digital camera to take an image in "real" world conditions, it is often 
found that the brightness of the image is not consistent across the image when measured 
from left to right. This can create problems with face image mapping as the brightness of 
the join line at each side of the head is at the same brightness. The join Une is the 3D 
surface line where the face portion of the head meets the sides of the face. If there is a 
difference in brightness at the join line, the result is a distinct and unwanted shade 
discontinuity on one or both sides of the mapped head. 

The paragraph beginning at page 26, Une 16: 



In order to combat this effect, "Automatic Shade Compensation" techniques are 
used during the mapping phase of the operation. This attempts to modify the brightness 
fifl^ of the 2D-captured image to give a more even mapping without unduly affecting the 
"look" of the overall image. 
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The paragraph beginning at page 26, line 20: 



This is achieved by applying a non-Unear filter to the 2D image, which modifies 
the intensity of each pixel as a function of pixel position and based upon the following 
equations. This filter is apphed to the whole image, however only pixels within the oval 
are used to calculate the filter parameters: 



The paragraph beginning at page 27, Une 3: 



Nsummed = Number of pixels used in the summation (i.e. Fall within 25% 
and 75% bounds) 



The paragraph beginning at page 28, line 11: 



Note that at x=Nx/2 no intensity changes occur on either side thus preventing a 
fflJ^ discontinuity from appearing at the center of the image. 



The paragraph beginning at page 28, line 19: 



Shape Head (S92) 

The "Shape Head" feature allows the user to alter the three dimensional shape of 
the current head as is shown in FIGURES 20 and 21. Control points allow the 
head to be squashed and stretched in dimensions coplanar to the face and an 
P( additional slider allows the depth of the head or scale in the front to back direction 

to be altered. 
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The paragraph beginning at page 30, line 4: 



Keep Changes (SlOO) 

Keeping changes updates the current selected slot with the alterations made during 
the editing session. Following this the user is again invited to save his or her work 
to a controller pack. 



i 



The paragraph beginning at page 20, line 8: 



The personalized heads that were created using the image capture/edit/save 
functions may be used during game play. Typically, a user will select a personalized 
head that includes the user's facial image and applies that head to the body of a desired 
animated game character. In this way, users can personalize game play by having the 
animated player that they control be a player that has the appearance of the user. During 
multiple-user combat game play, it would enhance the realism of game play if the 
animated game players appear in multi-player game mode. 



The paragraph beginning at page 30, line 16: 



Moreover, the personalize heads may be applied (in a random manner or some 
prescribed manner, e.g., all guards have a certain head) to computer controlled game 
characters, such as guards that appear in a combat game, where the guards are not 
controlled by any of the game playing users. During game play, any heads that occupied 
the "Single Player Slots" are randomly selected and used as the heads for that particular 
level's guard characters. Thus, the user will see personalized computer generated players 
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as he plays a game. In addition, if the slots set aside for multi-player games contain any 
valid personalized heads then these heads may appear (Fig. 14) as a head option when the 
player chooses to modify his character's properties. Thus, the player may use or edit any 
of the saved "heads" in the controller memory pack. 

The paragraph beginning at page 31, line 4: 

The 2D image and settings made with the editor are stored to save a personalized 
animated game player. The 2D image and settings may be stored in the user's controller 
memory pack so that they may be carried by the user from one game console to another. 
The memory pack plugs into the hand controller and may be carried by the user from 
game machine to game machine and, thus, allows the user to load his personalized 3D 
animated game player (including a face that is derived from a 2D image of the user) into 
any game machine that the user may be playing. Thus, some advantages of using 
( controller game packs for storage of personalized player head is that the user can carry 
personaUzed player settings to other game consoles, especially those consoles' owned by 
friends. Other advantages of a portable personalized player head stored in the controller 
memory pack include that friends may play each other where each friend has their own 
personalized player, and personalized images may be formed for the computer controlled 
players, e.g., automatically generated guards and other "bad guys" to randomly have 
personalized heads appear on animated figures during the game. 
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